IRREDUCIBILITY OF AUTOMORPHIC GALOIS REPRESENTATIONS OF 

GL(n), n AT MOST 5. 



FRANK CALEGARI AND TOBY GEE 

1 Abstract. Let tt be a regular, algebraic, essentially self-dual cuspidal automorphic representa- 

' tion of GL„(Aj^), where i<' is a totally real field and n is at most 5. We show that for all primes 

I, the Z-adic Galois representations associated to tt arc irreducible, and for all but finitely many 
, primes Z, the mod Z Galois representations associated to tt are also irreducible. We also show 

that the Lie algebras of the Zariski closures of the Z-adic representations are independent of I. 

1. Introduction 

■ 1.1. It is a folklore conjecture that the Galois representations (conjecturally) associated to alge- 
\ braic cuspidal automorphic representations of GL„(Ai?) over a number field F are all irreducible. 

(-H ' In general, rather little is known in this direction. Ribet ( |Rib77| ) proved this result for classi- 

cal modular forms, and his proof extends to the case of Hilbert modular forms ( |Tay95| ). The 
result was proved for essentially self-dual representations of GL3(Ai?), F totally real, in |BR92] . 
In [DV08j , Dieulefait and Vila proved big image results for a compatible family arising from a rank 
four pure motive M over Q with Hodge- Tate weights (0, 1,2,3), coefficients in a quadratic field 
K, and certain other supplementary hypotheses (see also |Die07| ). In a 2009 preprint ( |Ram09b] ) . 

■ Ramakrishnan proves irreducibility of the associated Z-adic representations for essentially self-dual 
representations of GL4(Aq) for sufficiently large I] his argument also applies without the assump- 
tion of self-duality assuming the existence of the corresponding Galois representations. 

\ Until recently, very little was known in the general case. It is sometimes the case that the 

■ Galois representations can be proved to be irreducible for purely local reasons; if the automorphic 
\ representation is square-integrable at some finite place, then it is a consequence of the expected 

local-global compatibility that the corresponding local Galois representation is indecomposable, 
which implies that the global Galois representation, being semisimple, is irreducible. In |TY07] . 
this observation was used to prove the irreducibility of the Galois representations considered 
in |HT01| whenever the square-integrability hypothesis holds. 
K> ■ One reason to suppose that the Galois representations should be irreducible is that if the 

^ I Fontaine-Mazur-Langlands conjecture holds, then (by a standard i-function argument) the re- 

" ■ ■ ducibility of an Z-adic Galois representation associated to an automorphic representation would 

show that the automorphic representation could not be cuspidal. In fact, it is enough to know 
that any geometric Galois representation is potentially automorphic, as this L-function argument 
is compatible with the usual arguments involving Brauer's theorem. This observation was ex- 
ploited in |BLGGTT0] to prove that if K is an imaginary CM field and tt is a regular, algebraic, 
essentially self-dual cuspidal automorphic representation of GL„(Ai^) which has extremely reg- 
ular weight (a notion defined in jBLGGTTO] ). then for a density one set of primes I, the ^-adic 
Galois representations associated to tt are irreducible. While this theorem is useful in practice, 
the condition that the weight of tt be extremely regular is sometimes too restrictive. For example, 
it is never satisfied by the base change of an automorphic representation over a totally real field 
if 71 > 3. In the present paper, we begin by extending the result of [ BLGGTTO] to the case of 
totally real fields if n < 5, with no assumption on the weight of tt. Just as in |BLGGTT0] . we use 
potential automorphy theorems and the L-function argument mentioned above. The key difficulty 
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with applying the potential automorphy theorems available to us is to show that any hypothetical 
summand of the Galois representation to be proved irreducible is both essentially self-dual and 
odd. It is here that the arguments of [BLGGTlO] make use of the condition of extreme regularity, 
which is not available to us. Instead, we observe that if n < 5 then any constituent of dimension 
at least 3 must be essentially self-dual for dimension reasons, and is then odd by the main result 
of [BC09| . One dimensional summands are trivial to deal with, which leaves us with the problem 
of dealing with 2-dimensional summands. However, any two-dimensional representation is essen- 
tially self-dual, so we need only show that we cannot have even two-dimensional constituents, at 
least outside of a set of places of density zero. To do this, we use a variant of the arguments 
of |CallO| . together with an argument using class field theory to show that there cannot be too 
many residually dihedral representations. 

The arguments outlined so far suffice to prove the result for a set of primes of density 1. In order 
to extend our result to all primes, we make use of a group-theoretic argument (in combination with 
our density 1 result) to show that the characteristic polynomials of the images of the Frobenius 
elements can only be divisible by the characteristic polynomials of a global character in certain 
special cases, which rules out the possibility of any of the Galois representations having a one- 
dimensional summand. We use the same argument to show that it is not possible for any of 
the representations to have a dihedral summand. We make use of the self-duality of the Galois 
representations we consider to reduce to these possibilities and so obtain the result. 

In order to extend this argument to the characteristic I representations, we show using class 
field theory that if infinitely many of the characteristic I representations have a one-dimensional 
summand, then the characteristic polynomials of the images of the Frobenius elements are divisible 
by the characteristic polynomials of a global character, which reduces us to the cases above. We 
prove a similar result for dihedral representations. This quickly reduces us to one special case, that 
of an irreducible 4-dimensional subrepresentation which when reduced mod I splits up as a sum of 
two irreducible 2-dimensional representations. In this case, we are able to exploit the connection 
between GO4 and GL2 x GL2 to reach a contradiction. 

Using similar arguments, we are also able to show that the Lie algebras of the Zariski closures 
of the images of the Z-adic representations are independent of I. Following |Ram09b] . we addi- 
tionally extend our analysis to the Galois representations associated to regular algebraic cuspidal 
automorphic representations of GL3 or GL4 over a totally real field which are not assumed to be 
essentially self-dual, under the hypothesis that the Galois representations exist. 

One may naturally ask whether these methods can be generalized to n > 6; we explain why 
this might be difficult. Suppose that tt is a regular algebraic cuspidal automorphic representation 
of GL3(Aq) which is not essentially self-dual (they exist!). Then, conjecturally, there should 
exist a compatible system of three dimensional Galois representations TZ = {r\{Tr)} of Gq. For a 
sufficiently large integer n, the compatible system TZ © (e" ^ TZ^) is a six dimensional compatible 
system of essentially self-dual regular Galois representations. Our method for ruling out that 
this (completely reducible) compatible system is associated to a regular, algebraic essentially self- 
dual cuspidal automorphic representation 11 of GL6(Aq) would consist of recognizing it as an 
isobaric sum tt ffl (tt^ (g) | • |") by proving the (potential) automorphy of a non-essentially self-dual 
representation rx{TT) : Gq GL3(Q/) for some prime /. However, such automorphy results are 
out of reach at present. 

We would like to thank Florian Herzig for helpful conversations about representation theory, and 
Dinakar Ramakrishan for making available to us the preprint [Ram09b] . We would also especially 
like to thank Robert Guralnick, who (together with Malle in |GM11| ) answered a difficult problem 
of the first author in the modular representation theory of finite groups. Even though (due to 
subsequent simplifications) we did not end up using this result, the mere knowledge of its veracity 
was helpful psychologically in the construction of several of our arguments. We would also like to 
thank Tom Barnet-Lamb, Kevin Buzzard, Luis Dieulefait, Matthew Emerton and Florian Herzig 
for their helpful comments on an earlier draft of this paper. 
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2. Preliminaries 



2.1. We recall some notions from |BLGGT10| . Let F be a totally real field. By a RAESDC 
(regular, algebraic, essentially self-dual, cuspidal) automorphic representation of GL„(Ai?), we 
mean a pair (tt, x) where 

• TT is a cuspidal automorphic representation of GL„(A^) such that tTqc has the same infin- 
itesimal character as some irreducible algebraic representation of the restriction of scalars 
from to Q of GL„, 

• X '■ A^/F^ — >■ is a continuous character such that X«(~l) is independent of i;|oo, 

• and Ti = tt"^ ® (^x° det). 

If f2 is an algebraically closed field of characteristic 0, we write (2")^°™^^'^'^+ for the set of 
a = {ar,^) e (Z")H°'°(^'") satisfying 

If o £ (2")^°™ let Sa denote the irreducible algebraic representation of GLn"''^'"^'''^^ which 

is the tensor product over r of the irreducible representations of GLn with highest weights Or- 
We say that a RAESDG automorphic representation (7r,x) of GL„(Ai?) has weight a if ttoo has 
the same infinitesimal character as (this is necessarily the case for some unique a). There is 
necessarily an integer w such that 

for aU T, i (cf. section 2.1 of jBLGGTlOj ]. 

We refer the reader to section 5.1 of [BLGGTIO] for the definition of a compatible system of 
Galois representations, and for various attendant definitions. If (tt, x) is a RAESDG automorphic 
representation of GL„(Aj7), then there is a number field M containing the images of all embeddings 
F ^ M and weakly compatible systems of Galois representations 

rxiT:) : Gp ^ GL„(Ma) 

and 

rx{x) -Gf-^mI 

as A ranges over the finite places of M (cf. the last paragraph of section 5.1 of IBLGGTlO] ). 
Suppose that tt has weight a e (2")Hom (f,c),-i-^ ^^^^ regard each element of Hom(F, C) as an 
element of Hom (F, AI). Then: 

• if S* is the finite set of finite places w of F at which ■Uy is ramified, then r\ (tt) and rx (x) 
are unramified unless v € S or v\l; 

• rA(7r) ^rA(7r)^ Oei-"rA(x); 

• if v\l then ^^(Tr)!^^^^ and '''a(x)Igf„ ^e Rham. If furthermore v ^ S then rx{Tr)\Gp^ 
and rx{x)\GF^ crystalline; 

• for each t : F ^ M and any M Ma over M, the set IITT-(rA(7r)) of r-Hodge-Tate 
weights of rxiir) is equal to 

{ars + {n- l),ar.2 + (n - 2), . . . , ar,n}- 

In arguments it will occasionally be useful to replace M with a finite extension, in order to compare 
two different compatible systems; we will do this without comment. 

If p ; G ^ GL(V^) is any semi-simple two dimensional irreducible representation which is 
induced from an index two subgroup G' of G, then, by abuse of notation, we call r dihedral. The 
image of a dihedral representation is a generalized dihedral group; equivalently, the projective 
image of p in PGL(T^) is a dihedral group. 

2.2. Oddness. We now recall from section 2.1 of jBLGGTTO) the notion of oddness for essentially 
self-dual representations of Gp- Let Z > 2 be a prime number, and let K ~ Qi or F;. If r : Gp — > 
GL„(_R') and fi : Gp — >■ are continuous homomorphisms, then we say that the pair (r, /x) 
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is essentially self-dual if for some (so any) infinite place w of F there is an e„ £ {±1} and a 
non-degenerate pairing (, ) on iiT" such that 

{x,y) = ev{y,x) 

and 

{r{(T)x, r{cy(TCv)y) = fJ.{cr){x, y) 
for all y £ K" and all a G Gp- Equivalently, (r, /i) is essentially self-dual if and only if either 
/i(ct,) = — e„ and r factors through GSp„(A') with multiplier fj,, or fj,{cy) — e„ and r factors through 
GOniK) with multiplier fi. 

We say that the pair (r, /i) is odd and essentially self-dual if it is essentially self-dual, and ev — I 
for all v\oo. 

Lemma 2.1. If (r, fi) is essentially self-dual and n is odd, then (r, ii) is odd. 

Proof. Since n is odd, r factors through GOn{K) with multiplier fi. Taking determinants, we see 
that for each w|oo, /^(ct,)" = 1, so that ^(c^) = 1, as required. □ 

We also have the following trivial lemma. 

Lemma 2.2. If x ■ Gf ^ is a character, then (XiX^) odd and essentially self-dual. 

We have the following important result of IBC09| . 

Theorem 2.3. Let {n,x) RAESDC automorphic representation o/GL„(Ai?), and denote the 
corresponding compatible systems of Galois representations by (r\{T:),r\{x))- If for some A we 
have an irreducible subrepresentation r of r\{Tr) with r = (g) e^^"r\{x), then {r,e^~"r\{x)) is 
essentially self- dual and odd. 

Proof This is Corollary 1.3 of |BC09| . □ 

Since rxij:) = ^^(Tr)^ ® e^~"rA(x)i if f is an irreducible subrepresentation of r\{T:) then there 
must be an irreducible subrepresentation r' of r\{'K) (possibly equal to r) with r' = r'^ ®€^^^r\{x)- 
In particular, we have: 

Corollary 2.4. Let (tt, x) be a RAESDC automorphic representation o/ GL„(Ai?), and denote 
the corresponding compatible systems of Galois representations by {r\{'K), r\{x)). If for some A we 
have an irreducible subrepresentation r ofr\{Tr) with dimr > n/2, then (r, e^^"f'A(x)) is essentially 
self-dual and odd. 

Proof. There is an irreducible subrepresentation r' of r>(7r) with r' = ® £^^"r\{x)', but dim r + 
dimr' > dimrA(7r), so we must have r' — r. The result then follows from Theorem 12.31 □ 

Suppose now that r : Gp GL2(Q;). Then r factors through GSp2(<Q;) with multiplier detr, 
so the pair (r, detr) is essentially self-dual and odd if det r(ct,) = —1 for all v\oo. We have the 
following variant on Theorem 1.2 of jCallO] . 

Proposition 2.5. Suppose that I > 7 is prime, and that r : Gp GL2(Q/) is a continuous 
representation. Assume that 

• r is unramified outside of finitely many primes. 

• Sym^ f|(3pj^^j is irreducible. 

• / is unramified in F. 

• For each place v\l of F and each t : Fy ^ Qi, WTr{r\cp^^) is a set of 2 distinct integers 
whose difference is less than (I — 2)/2, and rjc^ is crystalline. 

Then the pair (r, det r) is essentially self-dual and odd. 

Proof. Consider the representation s — Sym^ r. Since the pair (r, det r) is essentially self-dual, 
so is the pair (s,(detr)^). By Lemma [2.11 (s, (detr)^) is odd. By Corollary 4.5.2 and Lemma 
1.4.3(2) of |BLGGT10| . there is a Galois totally real extension F'/F such that (s|g^, , (det r)^ 1^^, ) 
is automorphic. By Proposition A of |TaylO| , for any place wjoo of i^' we have 

^rs\Gp,{Cv) = ±1, 
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so that 

detr\Gp,icv) = -1, 

and (r, det r) is odd, as required. 



□ 



2.3. Residually dihedral compatible systems. We now sliow that residually dihedral com- 
patible systems are themselves dihedral up to a set of places of density zero. 

Lemma 2.6. Suppose that I > 2 is unramified in F , and that s : Gp GL2(Q;) is a continuous 
irreducible representation, such that if v\l is a place of F, then 

• s\gp, is crystalline, and 

• for all embeddings F ^ Q;, HTt-(s|gj^^) consists of two distinct integers with difference 
less than (/ — 2)/2. 

Assume that s is dihedral, so that s is induced from a character of a quadratic extension K/F. 
Then I is unramified in K . 

Proof. Let v\l be a place of F. Assume for the sake of a contradiction that v is ramified in K. 
If s\qp^^ is irreducible, then s|gf„ induced from a character % of Gl, where L is the quadratic 
unramified extension of F^,. But then 

s|ga-„ = (IndG^"x)|GK„ - Indc^^^xICiA-, 

is irreducible, a contradiction. 

If on the other hand s\gp^ is reducible, then since it is isomorphic to the induction from 
of some character, it must be of the form tpi © ■!/)2 with ipitp2^ quadratic. Let k be the residue 
field of F^, and for each cr : fc ^ F/ let uJa be the corresponding fundamental character of Gp^ of 
niveau 1. By Fontaine-LafFaille theory, 

V'iV'2"'i/.„ = n 

where tto- is the (positive or negative) difference between the elements of WTt{s\gf^), where r : 
Fv ^ Q; is the unique lift of a. By assumption, we have 2ac, £ [2 — 1,1 — 2], so ('0iV'2'^)^l-fF„ 1; 
a contradiction. So I is unramified in K, as claimed. □ 

Proposition 2.7. Suppose that TZ is a regular, weakly compatible system of l-adic representations 
of Gp- Then there is a set of rational primes C of density one such that if A lies over a place 
of C and s is a two-dimensional irreducible summand of r\ such that s is dihedral, then s is also 
dihedral, and the pair (s,dets) is essentially self-dual and odd. 

Proof. Note firstly that if s is dihedral, then it is induced from an algebraic character of a quadratic 
extension of F . If this quadratic extension is not totally imaginary, then this character would be 
a finite order character times a power of the cyclotomic character, contradicting the regularity of 
s. So the extension must be totally imaginary, in which case det s(c„) = — 1 for all places ujoo of 
-F, and the pair (s, det s) is essentially self-dual and odd. 

Let S be the finite set of primes at which TZ ramifies, and let F' be the maximal abelian 
extension of F of exponent 2 which is unramified outside S (the extension F' /F is finite). By 
Lemma 12.61 for all but finitely many A, if s is as in the statement of the proposition then s\Gp, 
is reducible. Applying Proposition 5.2.2 of jBLGGTTO] to the regular weakly compatible system 
T^\Gpn we see that there is a set C of rational primes of density one such that if A lies over an 
element of C and s is as in the statement of the proposition, then s\gp, is reducible, so that s is 
dihedral, as required. □ 

3. Irreducibility for a density one set of primes 



3.1. In this section, we establish the irreducibility of rA(7r) for a density one set of primes A. 
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Proposition 3.1. Let F be a totally real field. Suppose that n is a RAESDC automorphic repre- 
sentation ofGhn{Ap), and that for some A we have a decomposition 

r\{Tr) = r-A(7r)i © ■ • • © rx{7r)j, 

where each rx{TT)i is irreducible. Suppose also that there is a totally real Galois extension F' / F 
such that each ^^(Tr)^!^'^, is irreducible and automorphic. Then j = 1, so r>(7r) is irreducible. 

Proof. This may be proved by an identical argument to the proof of Theorem 5.4.2 of [BLGGTTO] . 

□ 

Theorem 3.2. Let F he a totally real field. Suppose that (7r,x) is a RAESDC automorphic 
representation o/GL„(Ai?) with n<5. Then there is a density one set of rational primes C such 
that if A lies over a prime in C, then r\ (tt) is irreducible. 

Proof. Write 

r\[TT) = rx{Tr)i © • • • ®rx{Tr)j^, 
with each r\{Tr)i irreducible. By Proposition 5.2.2 of |BLGGTT0] there is a density one set of 
rational primes £ such that if A lies over a prime of C, then each f\{TT)i\Gj,^^^j is irreducible. We 
may assume that every prime in C is at least 13. 

If dimrA(7r)i > 3, then by Corollary 12.41 and the hypothesis that n < 5 we see that the pair 
{'''x{'^)i,e^~'^rx{x)) is essentially self-dual and odd. If dimrx{Tr)i = 1, then by Lemma [221 the 
pair {r\{TT)i,rx{n)^) is essentially self-dual and odd. 

Suppose now that di'nir\{'iT)i = 2. By removing finitely many primes from £, we see from 
Proposition 12.51 that we may assume that if A lies over an element of C, and Sym f\{TT)i\Gpi^^ ) 
is irreducible, then the pair (rA(7r)i, det rA(7r)i) is essentially self-dual and odd. If A lies over an 
element of jC and Sym f\{TT)i\G^^^^^ is reducible, then since r\{TT)i\GF((^-, irreducible, it follows 
from Lemmas 4.2.1 and 4.3.1 of [BLGGIO] that fx{TT)i has dihedral image. By Proposition 12.71 
after possibly replacing C with a subset of density one, the pair (rA(7r),i, detrA(7r)i) is essentially 
self-dual and odd. 

Thus if A divides a prime in C, for each i there is a character xxa such that the pair (rA (7r)i, XA,i) 
is essentially self-dual and odd. After possibly removing a finite set of primes from C, we may 
assume that every element of C is unramified in F, and that each rx{Tr)i is crystalline with 
Hodge- Tate weights in the Fontaine-Laffaille range. Fix some I G C and some X\l. Let K be an 
imaginary quadratic extension of F in which each place of F above I splits completely. By Theorem 
4.5.1 of jBLGGTTO] . there is a finite Galois CM extension K' of K such that each r\{TT)i\G^, is 
irreducible and automorphic. 

Let F' be the maximal totally real subfield oi K' . By Lemma 1.5 of |BLGHT09] each r\{TT)i\Gp, 
is irreducible and automorphic. The result now follows from Proposition 13. II □ 

4. Irreducibility for all primes 

4.1. In this section we prove that the representations rx{TT) are irreducible for all A. 

Theorem 4.1. Let F be a totally real field. Suppose that {tt,x) a RAESDC automorphic 
representation o/GL„(Af^) with n < 5. Then all of the representations rx{TT) are irreducible. 

By Theorem 13.21 rx{Tr) is irreducible for a set of A of density one. By Proposition 5.2.2 
of [BLGGTTO] ■ this implies that rxi^^) is irreducible for a set of A of density one. 

Definition 4.2. Let F be a number field. We say that a representation r : Gp GL„(Q;) is 
strongly irreducible if for all finite extensions E/F, rjc^ is irreducible. 

We would like to understand when the Galois representations rx{TT) which are irreducible can 
fail to be strongly irreducible. We begin with an easy group theory lemma. 

Lemma 4.3. Suppose that G acts irreducibly on a finite dimensional vector space V of dimen- 
sion n. Let G' be a normal finite index subgroup of G, and suppose that V\g' — ® W^fc decomposes 
non-trivially as a G' representation into m distinct irreducible representations. Then m\n and 
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there exists a proper subgroup H D G' of G of index m and an irreducible representation W of H 
such that ~ Ind^W^. 

Proof. Since the representations Wk are distinct, and since G" is normal, the group G acts tran- 
sitively on the set of representations Wk- In particular, all the Wk have the same dimension. Let 
W be one of these representations, and let H denote the stabilizer of W. By the orbit-stabilizer 
theorem, the index of in G is m. The representation W extends to a representation of H. Since 
Horn H(y,W) is non-trivial, by Frobenius reciprocity ilomciV,lnd'^W) is also non-trivial. Yet 
Ind^(W) has dimension [G : H] dim(W^) = dim(F) and V is irreducible, and thus the homomor- 
phism V — Ind^(W^) must be both an injection and a surjection, and hence an isomorphism. □ 

Using this lemma, we shall see that the density one set of irreducible Galois representations 
r\ (tt) remain irreducible upon restriction to any fixed finite extension, except in situations in which 
we can prove Theorem 14.11 directly. 

Corollary 4.4. Let F be a totally real field. Let (tt, x) be a RAESDC automorphic representation 
of GhniKp) with n < 5. Let X he a prime such that r\{Ti) is irreducible. Then either: 

(1) r\{'K) is strongly irreducible, or 

(2) (7r,x) is an automorphic induction, r\{Tr) is irreducible for all A, and f\{T:) is irreducible 
for all but finitely many A. 

Proof. We claim that for any finite extension E/F, either rx{TT)\Gj^ is irreducible or it decomposes 
into a sum of distinct irreducible representations. This follows immediately from the fact that 
'i^x{''^)\ge has distinct Hodge-Tate weights at any prime w\l. Suppose that r\{Tr)\Gj^ is reducible 
for some finite extension E/F. Replacing E by its normal closure over F, we may assume that 
the extension E/F is Galois, and hence by Lemma [4.31 we see that r\{TT) is the induction of an 
irreducible representation from some finite extension of degree dividing n. If n = 2, 3 or 5, the 
only possibility is that ^^(Tr) is the induction of a character from some degree n extension H of 
F. This character is de Rham, and is thus the Galois representation associated to an algebraic 
Hecke character. If n = 3 or 5, then [H : F] is odd, and thus H does not contain a CM field. It 
follows that the corresponding Galois representation is a finite order character times some power 
of the cyclotomic character. This contradicts the regularity of r\{T:). If n — 2, then H must be 
a CM field, and r\{Tr) is the induction of an algebraic Hecke character. The claims regarding the 
irreducibility of fxlir) are elementary to verify in this case. 

If n = 4, then cither ^^(Tr) is the induction of a character from some degree 4 extension L/F, or 
the induction of a two dimensional representation of some quadratic extension K/F. In the first 
case, since r\{TT) is regular, L contains a CM field. It follows that L must contain a subfield K of 
index two, and thus in both cases ^^(Tr) is the induction of some two dimensional representation of 
some quadratic extension K/F. It follows that there is an isomorphism ^^(Tr) ~ r\{'K)®7], where 77 
is the quadratic character of K/F. By multiplicity one for GL4(Ai?) ([JS8L) and by Theorem 4.2 
(p.202) of |AC89) . we deduce that {it,x) 

is an automorphic induction from some quadratic field 

K/F. 

It sufiices to prove that when n = 4 and (tt, x) is an induction of some cuspidal automorphic 
representation w of GL2(Ax) from some quadratic field K/F, then rxin) is irreducible for all A, 
and fxij:) is irreducible for all but finitely many A. If K/F is totally real, then vd corresponds 
to a Hilbert modular form with corresponding Galois representations s\{w), and there are iso- 
morphisms r\[TT) ~ Ind sa (tn) . The representation sx(m) is irreducible for all A, and s\{vj) is 
irreducible for all but finitely many A. If ^^(Tr) is reducible, then s\{vj) ~ s^('cc) where c is the 
non-trivial element of G'a\{K/F). Similarly, if f\(T:)\Qp is reducible for infinitely many primes A, 
then s\{w) ~ infinitely many A. In either case, by multiplicity one, we deduce that 

w ~ vo^, and by Theorem 4.2 (p. 202) of jAC89| . we deduce that w itself arises from base change 
from GL2(Ai?). If this is so, however, then tt is not cuspidal, contrary to assumption. Suppose in- 
stead that K/F is not totally real. By considering the possible infinitesimal characters associated 
to regular algebraic cuspidal automorphic representations of GL2(A;f), we see that tt cannot be 
regular. □ 
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In the sequel, it will be useful to collate some information about irreducible representations of 
semi-simple Lie algebras of small dimension. 

Proposition 4.5. Let k be an algebraically closed field of characteristic zero. Let G be the It- 
points of a reductive algebraic group acting faithfully and irreducibly on a vector space over k of 
dimension n. Let denote the connected component of G, let q be the Lie algebra of G^ , and 
write = 3 ® 1), with 3 is abelian and \) semisimple. Suppose that is not abelian. Then, for 
n < 6, \) is one of the following algebras, where the columns of the table correspond to whether G 
preserves a generalized orthogonal pairing, a generalized symplectic pairing, or does not preserve 
any such pairing respectively. 





GO 


GSp 


GL 


2 


* 


5I2 


* 


3 


5l2 


* 




4 


SO4 = SI2 X sb 


SI2, SI2 X Sl2, Sp4 




5 


5I2, SO5 = Sp4 


* 


sis 


6 


S06 


SI2, SI2 X Sl2, 5p6 


SI2 X sis, SI2 X SI2 X sl2, SI3 X sis, SI4, sle 



Proof. This can be checked directly by hand. In dimension n, the representation of 5(2 is the 
(n — l)st symmetric power of the tautological representation, which is orthogonal if n is odd and 
symplectic if n is even. The four dimensional symplectic representation of SI2 x sh is reducible, 
but the image of GL(2) x GL(2) in GL(4) is normalized by a group G which contains it with index 
two and does act irreducibly. The same is true of SI3 x SI3 (index two) and 5(2 x 5(2 x sb (index six) 
in dimension six. The algebra 5^ x SI2 has a six dimensional representation which is the tensor 
product of the standard representation of the first factor and the symmetric square of the second. 
Finally, SI4 has a six dimensional representation which is of the tautological representation. □ 

The key idea of our argument is that, with certain caveats, we can detect reducibility on the 
level of compatible systems. Suppose that TZ and S are two weakly compatible systems of Galois 
representations. 

Definition 4.6. Say that S weakly divides TZ if the characteristic polynomials of <S divide the 
characteristic polynomials of TZ. 

It is not true that if <S weakly divides TZ then there is a corresponding splitting of Galois 
representations. A good example to keep in mind is as follows. Suppose that tt is a RAESDC 
automorphic representation for GL2(Ai?) with central character tp which docs not have CM. Then 
if TZ and <S are the compatible systems associated to Sym"~^(7r) (g) tp and Sym"(7r) respectively 
then <S weakly divides TZ, but both compatible systems are irreducible. Nevertheless, we will be 
able to detect non-trivial information from weak divisibility. The key result is the following. 

Theorem 4.7. Let V be a vector space of dimension n < 5 over an algebraically closed field of 
characteristic zero. Let H C GL{V) be a Zariski closed subgroup, and assume that the connected 
component of the identity H° acts irreducibly on V. Suppose that every h G H has a fixed vector 
in V. Then H = PSL(2), n is odd, and V ^ Syni"^^(H^) where W is the standard representation 
o/SL(2). For a generic semi-simple element h e H, one has dim(F|/i =!) = !. 

Proof. Tautologically, H admits a faithful irreducible representation into GL(V), and thus H is 
reductive. Let Z be the center of H. By Schur's lemma, Z acts on V via scalars. Yet (by 
assumption) any z G Z G H has a fixed vector, and thus Z is trivial. In particular, H is semi- 
simple. It suffices to assume that every t G T C H has a fixed vector for every torus T on H. 
Since is connected, we may check this condition for on the level of Lie algebras. The only 
(semi-simple) Lie algebras f) which admit faithful irreducible representations of dimension at most 
five aresb, 5I3, SO4 = sh xsh, sp4 = 505, sU, andsl^. It is easy to check that the only possibility is 
that [) = sh- Since is connected with trivial center it must be the adjoint form of SL(2), which 
is PSL(2). The irreducible representations of PSL(2) arc given by the even symmetric powers of 
the standard representation of SL(2). The group H° = PSL(2) has a trivial outer automorphism 
group. Hence the action of conjugation hy h G H on is given by conjugation by a element 

8 



7 of PSL(2). It follows that j^^h acts trivially on PSL(2), and thus, by Schur's lemma, 7~^/i is 
a scalar, and so lies in the center Z of H. Yet we have seen that Z is trivial, and so it follows 
that H = . A generic semi-simple element in PSL(2) is conjugate to an element of the form 

h — ^-i^ : foi' which dim(y|/i = 1) = 1 unless z is a root of unity of sufficiently small 

order. □ 

Using this, we prove the following. 

Lemma 4.8. Let F be a totally real field. Let (7r,x) be a RAESDC automorphic representation 
o/GL„(Ai?) with n <b, and let TZ be the corresponding weakly compatible system. Suppose that 
either: 

(1) TZ is weakly divisible by a compatible system of algebraic Hecke characters. 

(2) For some finite extension E/F, "R^Ge weakly divisibly by a direct sum of compatible 
systems of two algebraic Hecke characters over E . 

Then either (tt, x) is automorphic induction, or n is odd, and there exists a finite Galois 
extension F' / F and a compatible system of two dimensional irreducible Galois representations S 
ofGp' such that TZlop, is a twist o/Sym"~ (S). Ln either case, r\{7T) is irreducible for all X, and 
fx{Tr) is irreducible for all but finitely many X. 

Proof. We may assume that (tt, x) is not an automorphic induction. By Corollary 14.41 we may 
find a sufficiently large A such that r\{TT) is strongly irreducible. We may also assume that fx^n) 
is irreducible. Let us assume that we are in case (1). After twisting, we may assuming that the 
compatible system TZ is divisible by the trivial character. Let H denote the Zariski closure of 
the image of r\{TT), and let iJ" denote the connected component of H. Since Frobenius elements 
are Zariski dense, we deduce that every h G H has a fixed vector. We deduce from Theorem 14.71 
that n is odd, and that the image of ^^(Tr) lands in the image of PSL2(Q/) in GO„(Q/) under 
the (n — l)-st symmetric power map. In particular, the image of rxijr) lands in SO„(Q;) and 
det(rA(7r)) ~ 1. The obstruction to lifting a projective homomorphism from PSL2(Q;) to GL2(Q/) 
lies in H^{Gf, Q; ), which vanishes by a result of Tate (for example, see Theorem 5.4 of [CM09] ). 
Hence there exists a Galois representation sx ■ Gp ^ GL2(<Q;) and an isomorphism 

^aW = Sym"-isA ®det(sA)^• 
We will show that sx is potentially modular, and use known irreducibility results for the Galois 
representations associated to Hilbert modular forms to conclude. 

We first show that ad''(sA) = Sym^(sA) det(sA)~^ is de Rham. We see this from the following 
plethysm for the standard representation W of GL(2): 

n-l 

(Sym"-^ W)'^^ ® det(I^)~("-i) = Sym^' W (g) det{W)-\ 

i=0 

In particular, ad°(sA) is a constituent of (rA(7r))®^, and hence is crystalline with Hodge-Tate 
weights in the Fontaine-LafFaille range for sufficiently large A. We claim that in fact ad°(sA) is 
regular; in the cases n — 1, 3 this is immediate from the regularity of rA(vr), and in the case n — 5 
it follows from the regularity of rx (tt) and the relation 

Sym2(ad"(sA)) -rA(7r)®l. 

Since rA(7r) is odd, it follows immediately that sx and hence ad''(sA) is also odd. Since it is 
2-dimensional, sx is automatically essentially self-dual, so that ad°(sA) is also essentially self- 
dual. If sx was dihedral or reducible, then fx{TT) would be reducible, contrary to assumption. 
Hence if A is sufficiently large, ad'^{sx)\GF((,) is irreducible, and so we deduce from Corollary 
4.5.2 and Lemma 1.4.3(2) of IBLGGTTO] that there is a finite Galois extension of totally real 
fields F' / F such that a,d'^{sx)\Gp, is automorphic. It follows that up to twist sa|g^, itself is 
automorphic (using the characterization of the image of the symmetric square in Theorem A and 
Corollary B of (RamOQaj ). say sx\gp, — rx{T^') for some tp. Since rx{Tr) is strongly irreducible, 
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r>(7r)|(3^, = Sym" ^ r\{n') C$idet{rx{n')) ^2" is irreducible, so tt' cannot be of CM type. Thus, for 

all A', we have rylcp, — Sym"^^ rx'iT^') ^ det{rx{'^'j)^~^ is irreducible, and fylcp, is irreducible 
for all but finitely many A'. 

Suppose instead that we are in case (2). After twisting, we may assume that the compatible 
system TZ\ge divisible by the trivial character. If the second character is also trivial (after this 
twist), we obtain a contradiction with Theorem 14.71 since the generic multiplicity of the h — I 
eigenspace is 1. Hence the characters are different. It follows as in the first paragraph of this proof 
that both the representations ^^(Tr) and rx(Tr) (E)x have trivial determinant for some x 7^ li which 
implies that x has finite order. Replacing E with the fixed field of the kernel of X: we reduce to 
the case that both characters are trivial, which is a contradiction. □ 

Corollary 4.9. Let F be a totally real field. Let (tt, x) be a RAESDC automorphic representation 
of GL„(Ai?) with n < 5. Suppose that some rx{Tr) is reducible, say rx{7T) — sx ® tx- Then 
min(dim s>,, dimt^) > 2, and neither of sx, tx can be dihedral. 

Proof. Suppose that without loss of generality dims a = 1. Since rx{T^) is de Rham, so is sa, so by 
e.g. Lemma 4.1.3 of |CHT08j there is an algebraic character x of Ap/F^ such that sx — rx{x)- 
Then the weakly compatible system {rx{x)} weakly divides {rxiir)}, so by Lemma l48l we see 
that ^^(Tr) is irreducible for all A, a contradiction. 

Suppose now that sx is dihedral. Then there is a quadratic extension E/F such that sx\ge is 
reducible, so is a sum of two de Rham characters. Arguing in the same way, we again obtain a 
contradiction from Lemma □ 

We now prove Theorem 14.11 We proceed by contradiction, assuming that for some A we have 
r\{TT) = Sx (B tx- By Corollarv 14. 9[ it suffices to consider the cases that both sx and tx are 
irreducible, that n = 4 or 5, and that dims^ = 2. 

4.2. The case n = 5. Since 5 is odd, we see from Lemma [2.11 that rx{Tr) factors through GO5 
with even multiplier. Since 2^3, we see that sx factors through GO2 with even multiplier, so sx 
is dihedral. This contradicts Corollarv 14.91 

4.3. The n = A symplectic case. Suppose that rx{T^) is symplectic with odd multiplier. If 
1^ is a (generalized) symplectic representation of a group G with multiplier Xi then there is a 
surjection f\^V x- Hence the virtual representation f\^{V) — x is an actual representation of 
G. In particular, if TZ is the compatible system of Galois representations associated to (tt, x), then 
A :~ A^(7^) — X is a compatible system of Galois representations such that ax ■ Gp — > GL5(Q;) 
has image in G05(Q;). Moreover, this compatible system is odd (automatic since 5 is odd) and 
regular. 

Since rxiir) — sx(Btx, we have 

ax®X\ = sx®tx<Si det{sx) © det{tx). 

In particular, as there are two characters on the right hand side, the representation ax contains a 
character, and we deduce that the compatible system A is divisible by the compatible system of 
a character. 

For a density one set of primes A', the representation ^^'(Tr) is irreducible. For such a prime A', 
let G denote the Zariski closure of the image, and G" the connected component of the identity. Let 
Z denote the center of H. If G^ acts reducibly, then r^/ is potentially reducible and hence (tt, x) 
is an automorphic induction by Corollarv 14.41 and we would be done. Hence G*^ acts irreducibly. 
Let g be the Lie algebra of G°, and let g = f) © 3 for f) semi-simple and 3 central. We deduce 
that [} acts irreducibly, and hence () — sp^ or SI2 d sp^ acting through the third symmetric power 
representation. In either case, the corresponding representation in dimension 5 of SO5 — sp^ is 
irreducible, and thus ax' is also irreducible, even after restricting to any finite extension E/F. 
Arguing as in the proof of Lemma [4.81 we deduce that there is a finite Galois extension F' /F of 
totally real fields and a RAESDC automorphic representation tt' of GL2(Ai?/) such that, for all 
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A", we have aylcp, = Sym^rA"(7r') det(rA" (tt')) ^. Since a>/ is irreducible, tt' cannot be of 
CM type, so that in fact ay is irreducible for all A". This contradicts the reducibility of a\. 

4.4. The n = 4 orthogonal case. Suppose finally that r\{TT) is orthogonal with even multiplier, 
and that r\{T:) — s\ Q) t\. By CoroUarv 14.91 both s\ and t\ are non-dihedral two-dimensional 
representations . 

Since r\{TT) factors through GO (4), it must either be the case that each of s\ and t\ factors 
through GO{2), or that the orthogonal pairing identifies s\ with t)( <^ £~^r\{x)- In the former case 
both sx and t\ would be dihedral, a contradiction, so we must be in the latter case. Since we also 
have t"^ = t\ ® det{t\)^^, we may write 

where tp = €~^r\{x) det{tx)~^- Since r\{'K) is de Rham, we see that tx and are de Rham. Thus 
"0 is pure of some weight. However, the representation rx{Tr) is pure, so if | Z is a finite place of 
F with TTy unramified, then all of the eigenvalues of rA(7r)(Frob^) are Weil numbers of the same 
weight. This implies that ip must be pure of weight 0; but this contradicts the regularity of rxiir). 

5. Representations with small image 

5.1. With an eye to proving that fx{TT) is irreducible for all but finitely many A, in this section, 
we prove a variety of results on residually reducible or dihedral 2-dimensional representations, 
using class field theory and the main conjecture of Iwasawa theory. 

Fix number fields F, M such that |HomQ(F, Af)| = [i^ : Q]. Fix a positive integer n and 
an element a G (Z")Hom (f,m) j^g-j- ^ j^g ^ place of F with residue characteristic I and residue 
field kx, and let pA : Gf Ghn{Mx) be a continuous dc Rham representation. Then we say 
that p has Hodge- Tate weights a if for each embedding t G Hom (F, M) inducing a place v of 
F via F ^ M ^ Mx, the r-Hodge-Tate weights of pa| Gf„ ^^'^ ar- Similarly, we say that a 
continuous representation Jj^ : Gf GLn{kx) is Fontaine-Laffaille of weight cr if / is unramified 
in F, and for each v\l, 'Px\gf^ admits a crystalline lift with Hodge- Tate weights a (equivalently, it 
has Fontaine-Laffaille weights determined by a in the usual way). 

Lemma 5.1. Let F/Q be a number field, and let a £ 2Hom(F,Af) ^^g^g^g fixed weight. Suppose 
that there exist infinitely many primes A such that Gp admits a character: xx '■ Gp k^^ with 
the following properties: 

(1) xa is Fontaine-Laffaille of weight a. 

(2) The conductor of ipx away from I is bounded independently of I. 

Then, for infinitely many X, there exists a finite order character (j) of Gp such that Xx4'~^ = V'A 
for a fixed algebraic Hecke character tp of weight a. If F does not contain a CM field, then ipx is 
a power of the cyclotomic character times a finite order character. 

Proof. The last sentence follows from the rest of the result by Weil's classification of algebraic 
Hecke characters. Note that since there are only finitely many finite order characters of Gi? with 
fixed ramification, it suffices to show that there is some algebraic Hecke character of weight a. 
Without loss of generality we may assume that M/Q is Galois, and then by (the proof of) Weil's 
classification, it is enough to check that for any g G Gal(Af/Q), ga (E) 1 annihilates Op (E) M.. 
Replacing a by gcr and each ipx by g^px , we see that it is enough to check that ct ® 1 annihilates 
0^; (8)IR. 

Regard each tpx as a character of Ap/F^ by class field theory. Since the ramification of the ipx 
outside of I is bounded independently of /, we see that there is a finite index subgroup U of Op 
such that each 4'\\u is just the composition of a and reduction mod A. Thus, for any u £ U, we 
see that (cr(u) — 1) is divisible by A; since this holds for infinitely many A, we see that o'{u) = 1. 
Since U has finite index in Op, the result follows. □ 

Corollary 5.2. Let F be a totally real field. Suppose that there are infinitely many primes I 
and 2-dimensional dihedral representations sx '■ Gp GL2(fcA) of fixed distinct Fontaine-Laffaille 
weights and fixed tame level. Then: 
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(1) For all but finitely many I, s\ is induced from a quadratic CM extension F'/F unramified 
at I. In particular, s\ is (totally) odd. (The field F' may depend on I.) 

(2) For infinitely many I, s\ is the reduction of the Galois representation associated to a fixed 
RAESDC automorphic representation tt^ o/GL2(Ai^) which arises from the induction of 
an algebraic Hecke character for Ap, for some CM extension F'/F. 

Proof. Each s a is induced from some quadratic extension F\/F unramified outside of I and a fixed 
set of places. By Lemma [2.61 for all but finitely many A, F\/F is unramified at places dividing 
I. Thus there are only finitely many possible extensions F\/F, so it suffices to show that any 
extension F'/F which occurs infinitely often is CM. However, if F' is not CM, then it does not 
contain a CM subfield, and Lemma [5. f I (applied to the characters of Gp' from which the s\ are 
induced) contradicts the assumption that s\ has distinct Fontaine-Laffaille weights. 

The second part now follows from Lemma l5.f l in the same way. □ 

Lemma 5.3. Let F be a number field. Then there are only finitely many A such that there exists 
an irreducible representation 

P^-.Gf-^ GU(kx) 

such that p\ has fixed distinct Fontaine-Laffaille weights, and'px has projective image A^, orA^. 
More generally, there are only finitely many A such that there exists an irreducible representation 

P^:Gf^ GL„(fcA) 

such that p\ has fixed distinct Fontaine-Laffaille weights, and "p^ has projective image of bounded 
order (independent of X). 

Proof. Suppose to the contrary that there are infinitely many such representations. By Fontaine- 
Laffaille theory, we see that the order of the projective image of grows at least linearly with I; 
but it is also bounded, a contradiction. □ 

Lemma 5.4. Let F be a totally real field. Then there are only finitely many A such that there 
exists an irreducible representation 

such that det(px{cy)) is independent of any infinite place v of F, ad*'(pA) is crystalline with fixed 
distinct Hodge-Tate weights and fixed tame level, and'px is reducible. 

Proof. Assume not, so that infinitely many such representations exist. We may assume that I 
is odd, unramified in F and is sufficiently large that ad*'(pA) has Hodge-Tate weights in the 
Fontaine-Laffaille range. By Lemma 15.11 we may deduce that, for infinitely many A, there is an 
isomorphism ijixT^ — "^^^ ® for a fixed integer fc ^ and a fixed finite order character 

ip. By Fontaine-Laffaille theory, we deduce that ad°(pA) is ordinary at all primes v\l. 

Suppose that "py^ = i/'w'^c/) © <j). Applying Ribet's lemma, we obtain integral lattices in px 
which give nonzero classes in the groups Ext (i/'w'^'/', 0) and Ext (0, t/''^'^?!')- Consider the 
corresponding lattices in ad°rA. Looking at the "top extension" in ad"(pA), we obtain classes in 
the groups 

H\F,uj-''i^-^). 

These classes are nonzero since I ^ 2, and are unramified outside N , I, and oo by construction. 
Moreover, in the second case, by the ordinarity of a.d^{px) the class is also unramified and conse- 
quently trivial at all v\l. If M is a Gi^-module, we may define a set of Selmer conditions as follows. 
Let C = {Cv} where Cv C H^{Gv,M) is defined to be: 

(1) C,=H\GJI,,Mi-) {iv\l. 

(2) Cy =0 if 

We note the following: 

Proposition 5.5. Fix a place v, an integer m ^ {0, 1}, and a finite order character x- Let lj 
denote the mod-l cyclotomic character. Then H^{Fy,ui"'^ • x) = fof sufficiently large I. 
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Proof. We may assume that v \ I. Let q — N{v) and let p be the residue characteristic of v. There 
is an equahty 

\H\F,,M)\ = \H\F,,M)\\H^{F,,M)\ = \H''{F,,M)\\H\F,,M*)\. 

Hence, if H^{Fy,Lu"^ ■ x) is non-trivial, then x is unramified at u, and 

t^"(Frob„)x(Frob„) £ {l,q}. 

Since aj(Frob„) = q, it follows that x(Frobi,) G {g^™, q^^™}. If x has order d, it follows that 

(g"^" - l)((7''(™-i) - 1) = (mod 0- 

Since ni ^ {0, 1}, this equality can only occur for finitely many I. □ 

It follows that for sufficiently large / the classes constructed above lie in the Selmer groups 
H}^,{F,u!^ip) and H}^{F,u!~^ip~'^) respectively, where C* is the dual Selmer condition (with no 
restriction on the class at v\l), with the possible exception of the class in H^{F,uji/j) when k — 1. 
We now consider three cases. 

(1) Suppose that lu'^iIj is (totally) odd. Then the main conjecture for totally real fields as 
proven by Wiles jWilQOj shows that (for I odd) / divides \H^{F,Lu^''ip~^)\ if and only if I 
divides 

L{l,Lu-''iJ;-^) =L{l-k,ilj-^)^0. 

(2) Suppose that uj'^'ip is even and fc > 1. Then, by Theorem 2.19 of jDDT97| (the global 
duality formula for Selmer groups, which is a refiection formula in this case), we deduce 
that 

\Hi,{F,u'^i^)\ = \Hl{F,oj^-''^r% 
Once more by Wiles this group is non-trivial if and only if I divides 

(3) Suppose that uj'^ip is even and fc = 1. Then we still have a class in H^{F,Lu^^ip^^). Let 
E = F{ip). Then, by restriction, we obtain a class in 

Hl{F,u:-'^-') ^ Hl{E,u-^) ^ ^ (i?(0), F;)""' ■ 

The latter group is isomorphic to the cj^^-part of the Z-torsion of the class group of E{(i). 

Yet, by Theorem 5.4 of |Keu89j . the uj~^ part of this group injects into K2{Oe)/1- Since 

K2{Oe) is finite, this group is trivial for / sufficiently large. 
In each case, we deduce that I divides a fixed non-zero rational number which is independent of 
and hence / is bounded. □ 

6. Residual irreducibility for all but finitely many primes 

6.1. We now bootstrap our previous arguments to prove the following result. 

Theorem 6.1. Let F he a totally real field. Suppose that (7r,x) is a RAESDC automorphic 
representation of Gh„{Ap) with n<5. Then all but finitely many of the residual representations 
f\{Tr) are irreducible. 

We firstly establish the following corollary of Lemma 14.81 and Corollary 14.91 

Lemma 6.2. Let F be a totally real field. Let (7r,x) be a RAESDC automorphic representation 
of GL„(Ai?), n < 5, and let TZ = {''a ('"')} be the associated weakly compatible system. Suppose 
that, for infinitely many primes X, at least one of the following holds: 

(1) f\{T:y^ contains a character. 

(2) f\{T:y^ contains a two dimensional dihedral representation. 
Then, respectively, at least one following also holds: 

(1) TZ is weakly divisible by a compatible system of algebraic Hecke characters. 

(2) For some finite extension E / F , TV\Ge i^ weakly divisibly by a direct sum of two compatible 
systems of algebraic Hecke characters. 
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In particular, rx{Tr) is irreducible for all but finitely many A. 

Proof. Denote the corresponding sub- representations of f\{TTY^ by s\. If the Hodge- Tate weights 
of rx^n) are in the Fontaine-LaffaiUe range, then there are a fixed number of possible Fontaine- 
Laffaihe weights of s\, which are independent of A. Similarly, there are finitely many possible Serre 
levels determined by the auxiliary ramification structure of TZ. The result follows by Lemma 15.11 
and Corollary 15.21 respectively (with the last sentence following from Lemma 14.81 and Corollary 

USD. □ 

We now prove Theorem 16.11 Assume for the sake of contradiction that there are infinitely 
many A with fx^T:) irreducible. By Lemma 16.21 there can only be finitely many A such that 
^xi^^T^ contains a character. This is already a contradiction when n < 3, and when n = 4 or 
n = 5 it implies that there are infinitely many A for which fx{TTY^ = sx (1) tx with sx and tx both 
irreducible, dims> — 2, and neither of sx and tx are dihedral. 

6.2. The case n = 5. Since 5 is odd, we see from Lemma [2.11 that rxiir) and thus fxin) factors 
through GO5 with even multiplier. Since 2 ^ 3, we see that sx factors through GO2 with even 
multiplier, so sx is dihedral. This can only happen for finitely many A by Lemma 16.21 



6.3. The n — A symplectic case. We argue as in section l473l Let TZ be the compatible system 
of Galois representations associated to (7r,x), and define A := /\^{TZ) — Xi a compatible system 
of Galois representations such that ax : Gp ^ GL5(Q;) has image in G05(Q;). Again, this 
compatible system is odd and regular. 

Since fx (tt) = ® , we have 

ax®X\ = sx'E)tx® det{sx) ® det(tA)- 

In particular, as there are two characters on the right hand side, the representation ax contains a 
character for infinitely many A, and as in the proof of Lemma |6.2[ we deduce that the compatible 
system A is weakly divisible by the compatible system of a character. 

Arguing as in section l473l we deduce that there is a finite Galois extension F' /F of totally real 
fields and a RAESDC automorphic representation tt' of GL2(Ai;'/), which is not of CM type, such 
that, for aU A, we have axlop, — Sym^rA(7r') det(rA(7r'))~^. Then for all but finitely many A 
the projective image of fxiT^') contains PSL2(F;) and axlcp, is irreducible, a contradiction. 

6.4. The n = 4 orthogonal case. It will be useful in the sequel to exploit the exceptional 
isomorphism of Lie groups SO4 ~ 5(2 x 5(2. More precisely: 

Lemma 6.3. Let F be a number field. Suppose that r : Gp ^ G04(Q;) is a continuous represen- 
tation. Then either: 

(1) there are continuous representations a, b : Gf ^ GL2(Q;) with r = a®b, or 

(2) there is a quadratic extension K/F and a continuous representation a : Gk ^ GL2(Q;) 
with r\cK = a® a'^, where Gal {K/F) = {1, c}. 

Proof. We have an exact sequence 

^ ^ GL2(Q,) X GL2(Q,) ^ G04(Q,) ^ {±1} ^ 

(cf. section 1 of |Ram02| ). Suppose firstly that the composite r : Gp — > G04(Q;) {±1} is 
not surjective. Then the obstruction to lifting r to a homomorphism Gp GL2(Q;) x GL2(Q;) 
lies in H'^{Gf,Q^), which vanishes by the proof of Theorem 5.4 of |CM09] . If the composite 
r : Gp — > G04(Q;) —5- {±1} is surjective, then we let Gk be the kernel of this composite, and the 
result follows as in the previous case. □ 

By Lemma |6.3[ we may assume either that for infinitely many A we have = ca ® &a for some 
ax, bx ■ Gp GL2(Q/), or that for infinitely many A there is a quadratic extension K = Kx/F 
with r\GK = ax® ttx for some ax : Gk GL2(Q;). 
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Suppose that we are in the first case. If Sa and bx are both reducible, then the semi-simplification 
of a\ €5 6a consists of four characters, contrary to assumption. Hence, without loss of generality, 
we may assume that 6a is irreducible infinitely often. There is an isomorphism 

A^rA(7r) ® (detrA(7r))"^ ~ ad°aA © ad''6A, 

from which we see that ad'^aA and ad'^6A are both de Rham, and are crystalline for all but finitely 
many A. They are both regular (this can be seen from a consideration of the Hodge-Sen- Tate 
weights of ax and 6a). 

After passing to a subset, we may assume that the Hodge- Tate weights of SLd^{a\) and a,d^{bx) 
are independent of A. Assume firstly that a a and 6 a are irreducible for infinitely many A. 

By Lemma 15.31 and the classification of finite subgroups of PGL2(F;) (cf. Theorem 2.47 of 
[DDT97p . we may assume that a\ and 6a are either dihedral or have images containing SL2(Fi). 

If both OA and 6a have image containing SL2(F;), then a a (^b\ is either irreducible or breaks 
up as a sum of a character and a 3-dimensional representation, a contradiction. If without loss of 
generality a a is dihedral and 6a has image containing SL2(F/), let K/F be the quadratic extension 
from which ax is induced. Then bxlcK is irreducible, so ax <8i 6a is irreducible, a contradiction. 
The remaining case is that a a and 6a are both dihedral. Then rA(7r) is completely decomposable 
over some quartic extension, which implies that sx and ^a are both dihedral, a contradiction. 

We may thus assume that for infinitely many A, oa is reducible and 6a is irreducible. If 6a is 
dihedral then one of sx and tx is dihedral, which can only occur finitely often, so by Lemma 15.31 
(applied to ad^bx) we may assume that the image of 6a contains SL2(F/). Then for A sufficiently 
large ad'^bx is irreducible, so as in the proof of Proposition 12.51 we see that ad ''6a is potentially 
automorphic and 6a is odd. Since the multiplier character of rx (tt) = oa CE> 6a is even, we see that 
detaA(c„) is independent of v\oo. Then Lemma 15.41 implies that there are only finitely many A for 
which such an a a can exist. 

This contradiction means that we may assume that we are in the second case, so that for 
infinitely many A, there is a quadratic extension K/F (which might depend on A) and a continuous 
representation a a : Gk — > GL2(Q/) such that rA(7r)|G^^ = ca (8> 

We claim that a a is necessarily dihedral for all but finitely many A G S*. We prove this by 
eliminating the other possibilities. Firstly, ax cannot be reducible, because if oa = ® Xi then 

OA «) Ca = «) (^'^ © X ® x"" ffi ® X'' © X "X) 0^ 

and the first two characters descend to Q, so that fx{7r) would have one-dimensional subrepresen- 
tations. 

If ax has projective image A^, Si or A-^, then the projective image of fx{Tr) is bounded inde- 
pendently of A. By Lemma 15.31 ttiis can only happen for finitely many A. 

The image of ax cannot contain SL2(F^). If it did, then ax ($> ax would either be irreducible or 
a sum of an irreducible three-dimensional representation and a character, depending on whether 
the projective representation associated to ax extends to F or not. This again contradicts the 
assumption that fxijr) is a sum of two irreducible 2-dimensional representations. 

Having eliminated the other possibilities, we see that for infinitely many A, ax is dihedral. Then 
for infinitely many A, sx and ^a each become reducible over quartic extensions of F, and are thus 
dihedral. This contradiction completes the proof. 

7. Lie algebras 

In this section we prove that the Lie algebras of the images of the rx{Tr) are independent of A. 
More specifically, we prove the following theorem. 

Theorem 7.1. Let F be a totally real field. Suppose that (7r,x) is a RAESDC automorphic 
representation of GJjn{Ap) with n < 5. Let Gx denote the Zariski closure of the image rx{Tr){GF), 
and let flA = 3a ffi f)A denote the Lie algebra of Gx, where ja is abelian and t\x is semisimple. Then 
Qx is independent of X, andtjx is either s\2, SO4 or sp4 (ifn = 4), or SO5 = sp4 (ifn = 5). 
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Proof. The result is trivial if n = 1 and standard if n = 2, so we may assume that n > 3. If 
(tt, x) is the automorphic induction of a character, then certainly g\ is abelian and independent 
of A. If n = 4 and (tt, x) is the automorphic induction of a RAESDC automorphic representation 
of GL2{Api) for F' I F & quadratic totally real extension, then either this representation is of CM 
type, and (tt, x) is the automorphic induction of a character, or is independent of A and f)A is 
equal to 5(2 (acting reducibly). 

Excluding these cases, by Corollary 14.41 (and its proof) we may assume that (7r,x) is not an 
automorphic induction, and that r\('K) is strongly irreducible. In general the independence of 3a 
of A is an easy consequence of Schur's lemma. Therefore we need only determine Suppose 
firstly that the compatible system {r\{-K)} is weakly divisible by a compatible system of algebraic 
Hecke characters. Then by Lemma [4.81 we see that n = 3 or 5, and that ()a = sb, acting through 
the {n — l)st symmetric power representation, independently of A. 

Conversely, if n = 3 or 5 and for some A we have ^\ = 5(2 acting through the (n — l)st symmetric 
power representation, then we claim that the compatible system {^^(Tr)} is weakly divisible by a 
compatible system of algebraic Hecke characters. To see this, write G for the Zariski closure of 
r\{'K)(G f), and G° for the connected component of the identity. Then the derived subgroup of 
must be PSL2, and since PSL2 has no outer automorphisms, Schur's lemma shows that G is 
necessarily of the form Z{G) x PSL2. Since Z{G) acts via a character (again by Schur's lemma), 
we see that the compatible system {t\{'k)} is weakly divisible by a compatible system of algebraic 
Hecke characters, as required. 

Examining the table in Proposition 14.51 '^e see that we are done unless n — A. In this case if 
X is odd then each r\{'K) has even multiplier and is thus orthogonal, and we see from the same 
table that \)\ = SO4 for all A. If x is even then for each A either \)\ = 5I2 (acting via Sym^) or 
t)A = SP4. We distinguish between these two possibilities by arguing as in Section l473l Consider 
the compatible system A :— /\^{TZ) — x- This is a compatible system of odd, regular Galois 
representations such that a\ : Gp ^ GL5(Q;) has image in G05(Q;). If for some A we have 
f)A = 3(2 then as above the compatible system A is weakly divisible by a compatible system of 
algebraic characters, and the argument of the proof of Lemma FOl shows that \)\ =3(2 for all A, 
as required. □ 

8. NON SELF-DUAL REPRESENTATIONS OF GL3 AND GL4 

In this section, we follow |Ram09b] and sketch a proof that our earlier irreducibility results 
extend to the case of regular algebraic cuspidal automorphic representations tt of GL3(Ai?) or 
GL4(Ai;'), F a totally real field, without assuming that tt is essentially self-dual, but with the 
assumption that the Galois representations rxi^-K) exist. 

Assume throughout this section that F is totally real, that n = 3 or 4 and that there is a weakly 
compatible system [rxij^y] of Galois representations associated to tt. We will demonstrate the 
required irreducibility results by reducing to the essentially self-dual case. 

Note firstly that the proof of CoroUarv 14.41 made no use of the essential self-duality of tt, so we 
have the following. 

Lemma 8.1. Lei X be a prime such that rxl?:) is irreducible. Then either: 

(1) r\{T:) is strongly irreducible, or 

(2) TT is an automorphic induction, rx{TT) is irreducible for all X, and fxi^i:) is irreducible for 
all but finitely many X. 

The following Lemma and Corollary will be our main tool to reduce to the essentially self-dual 
case. 

Lemma 8.2. Suppose that r : Gp ~> GL4(Q;) is strongly irreducible, and suppose that A^r : 
Gp ^ GLg(Q;) is not strongly irreducible. Then r ~ r^x for some character x- 

Proof. This is a standard argument (cf. Theorem 6.5 of jAR07| ). Consider the Zariski closure 
G of the image of r. Let G^ denote the connected component of G, let g be the Lie algebra of 
G°, and write g = 3 © f), with 3 is abelian and () semisimple. By assumption, G° acts irreducibly 
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in dimension 4. If A^r is not strongly irreducible, then f) = 5(2, SO4, or sp^. It follows that G" 
preserves a symplectic or orthogonal form, from which it is easy to deduce (using, for example, 
the facts that the normalizers of Sp4 and SO4 in GL4 are respectively GSp4 and GO4) that the 
image of r is symplectic or orthogonal, as required. □ 

Corollary 8.3. Suppose that n = 4, that n is not essentially self-dual, and that for some X, r\(Tr) 
is strongly irreducible. Then A'^r\{7r) is strongly irreducible. 

Proof. Suppose that A'^rxiir) is not strongly irreducible. By Lemma 15^ we see that r;^(7r) ~ 
r>(7r)^X> for some character x\] but then strong multiplicity one for GL4 implies that tt is essen- 
tially self-dual, a contradiction. □ 

Lemma 8.4. If n ~ 4, then it is impossible for r\{TT) to have a 1-dimensional summand. 

Proof. This may be proved in exactly the same way as Proposition 7.8 of |Ram09b] . Suppose that 
rxi"^) — s\ with xa a character. Then xa and detSA are both algebraic characters of Gf, 

so arise from automorphic representations x and v of GLi(Ai?). One easily obtains an equality of 
incomplete i-functions 

Since A^(7r) is an isobaric automorphic representation of GL6(Ai?) by |Kim03| . we see that the 
left hand side is holomorphic at s = 1, but the right hand side has at least a simple pole at s = 1, 
a contradiction. □ 

Lemma 8.5. For a density one set of primes X, rx{7T) is irreducible. 

Proof. Suppose not. By Lemma [8.4) there is a set of primes A of positive density such that rx^n) 
decomposes as a sum of irreducible representations of dimension at most 2. The result now follows 
by the same proof as Theorem 13.21 □ 

Theorem 8.6. ^^(Tr) is irreducible for all X. 

Proof. Let TZ denote the compatible system {r\{TT)}. By Theorem 14.11 we may assume that tt 
is not essentially self-dual. By Lemmas 18.11 and 18. 5[ and Proposition 5.2.2 of |BLGGTT0] . we 
may assume that r\{TT) is strongly irreducible for some A. Then the proof of Lemma [4.81 goes 
through verbatim, as does that of Corollary 221 and we see that it is impossible for rx> (tt) to have 
a one-dimensional summand for any A'. 

We are done if n = 3. If n = 4, the only possibility is that for some A', rx>{TT) = sy ® ty, 
with s\ and t\ both 2-dimensional. Since we have assumed that tt is not essentially self-dual, 
we see from Corollarv 18.31 that (using the same A as in the first paragraph) A'^r\{TT) is strongly 
irreducible. On the other hand, 

AVA'(7r) = sy (g)ty ©det(sA') ©det(iA')- 

It follows that the compatible system a'^TZ is weakly divisible by the compatible system of a 
character (in fact, two characters). After twisting, we may suppose that this character is trivial. 
We then obtain a contradiction as in the proof of Theorem 14.71 (note that the semisimple part of 
the Lie algebra of the Zariski closure of A^rA(7r)(Gi?) is 5I4 acting via A^, and it is not the case 
that every element of SI4 fixes some vector under this action). □ 

Theorem 8.7. For all but finitely many A, f\{Tr) is irreducible. 

Proof. By Theorem 16. 1[ it is enough to assume that tt is not essentially self-dual. Again, the proof 
of Lemma 16.21 goes over without change to the present setting, completing the proof if n = 3. If 
n = 4, it suffices to show that there cannot be infinitely many A for which f\{T:) = s\ (St\ with 
sx and ^a 2-dimensional. However, we again note that in this case we have 

A'^rxin) = SA ® ^A © det(sA) © det(fA), 

and we deduce that the compatible system {A^rA(7r)} is weakly divisible by the compatible system 
of a character. This gives a contradiction as in the proof of Theorem 18.61 □ 
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Theorem 8.8. Let Qx Lie algebra of the Zariski closure ofr\{Gp). Then Q\ is independent 

ofX. 



Proof. As in the proof of theorem Theorem [7jTJ if we write 0a = t)A © 3a with [)a semisimple and 
3a abelian, it suffices to show that [}a is independent of A. 

If n = 3, then the proof of Theorem 17.11 goes through unchanged, so we may assume that 
n = 4. By Theorem lT-l) we may assume that tt is not essentially self-dual, and by Lemma l8. II and 
Theorem 18.61 we may assume that rA(7r) is strongly irreducible for all A. It then follows from the 
proofs of Lemma [8?2l and Corollary 18. 31 that t)\ — sh for all A, as required. □ 
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